Method and system for zoomable attachment handling on a portable electronic device

ABSTRACT

A method for viewing a full image of a server stored original attachment on a portable electronic device including: building a graph structure within the server representing a map of the original image, downloading and displaying a re-sized image from the server to the portable electronic device, the original image exceeding an image size limit of the portable electronic device indicative of screen size of the portable electronic device, dividing the original image into four individual quadrants and calculating width and height parameters of the four individual quadrants, sending a request from the portable electronic device to the server to enlarge the image displayed on the device based on the width and height parameters, the request including the image size limit, retrieving and traversing the graph structure within the server to locate a separate image component constructed for the image size limit, collecting image binary data from the separate image component and modifying the image binary data based on the width and height parameters to create the full image of the original image, downloading each of the four resized quadrants to the attachment viewer and assembling the resized quadrants to display the full image of the original image on the portable electronic device.

FIELD

The present application relates to displaying attachments on a portableelectronic device and, in particular, a method for requesting andviewing an attachment image on a portable electronic device.

BACKGROUND

Wireless technology has made it possible for email messages to bereceived and displayed by various portable electronic devices includingPersonal Digital Assistants (PDAs), for example. Sending email messagesincluding attachments, such as photographs or scanned documents, isbecoming increasingly popular, however, most attachments are formattedfor a rich desktop or printer experience and generally require a desktopPC with a large screen display for proper viewing. As such, viewingthese attachments on the small screens of most portable electronicdevices presents a challenge.

Currently, when a user makes a request to view an attachment, therequest is received at a server where the attachment is scaled down tofit on the portable electronic device screen and then returned to theportable electronic device. When viewing high resolution attachments,such as scanned documents, for example, details of the scaled down imageare often difficult to distinguish. In order to view text or otherdetails of the scaled down image, the user may enlarge a desired portionby selecting an appropriate menu option. Each time the user requestsenlargement of an area of the scaled down image, the request is sent tothe server and a higher resolution image of the selected area isreturned to the portable electronic device. Often, the enlargementprocess must be performed multiple times before the user is able to viewthe details of the entire scanned document.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments will be better understood with reference to thefollowing figures in which like numerals denote like parts and in which:

FIG. 1 is a schematic diagram of a wireless communication system;

FIG. 2 is a block diagram of components of a portable electronic deviceaccording to an embodiment;

FIG. 3 is a flowchart depicting device side operation for viewing anattachment on the portable electronic device of FIG. 2;

FIG. 4 is a screen shot of the portable electronic device of FIG. 2;

FIG. 5 is another screen shot of the portable electronic device of FIG.2;

FIG. 6 is a flowchart depicting server side operation for viewing anattachment on the portable electronic device corresponding to the deviceside flowchart of FIG. 3.

DETAILED DESCRIPTION

In one aspect there is provided a method for viewing a full image of aserver stored original attachment on a portable electronic deviceincluding: building a graph structure within the server representing amap of the original image, downloading and displaying a re-sized imagefrom the server to the portable electronic device, the original imageexceeding an image size limit of the portable electronic deviceindicative of screen size of the portable electronic device, dividingthe original image into four individual quadrants and calculating widthand height parameters of the four individual quadrants, sending arequest from the portable electronic device to the server to enlarge theimage displayed on the device based on the width and height parameters,the request including the image size limit, retrieving and traversingthe graph structure within the server to locate any corresponding imagecomponent for the original image and, upon locating the correspondingimage component iterating through attributes of the image component todetermine if a separate image component has been constructed for theimage size limit, in the event that the separate image component has notbeen constructed then collecting image binary data from the originalimage within the server, constructing a new image component from thebinary data and caching the new image component as an attribute of theoriginal image in the graph structure, collecting further image binarydata from one of either the new image component or the separate imagecomponent and modifying the further binary data based on the width andheight parameters to provide four resized quadrants of the originalimage, downloading each of the four resized quadrants to the attachmentviewer and assembling the resized quadrants to display the full image ofthe original image on the portable electronic device.

In another aspect there is provided a method for viewing a full image ofa server stored original attachment on a portable electronic deviceincluding: building a graph structure within the server representing amap of the original image, downloading and displaying a re-sized imagefrom the server to the portable electronic device, the original imageexceeding an image size limit of the portable electronic deviceindicative of screen size of the portable electronic device, dividingthe original image into four individual quadrants and calculating widthand height parameters of the four individual quadrants, sending arequest from the portable electronic device to the server to enlarge theimage displayed on the device based on the width and height parameters,the request including the image size limit, retrieving and traversingthe graph structure within the server to locate a separate imagecomponent constructed for the image size limit, collecting image binarydata from the separate image component and modifying the image binarydata based on the width and height parameters to create four resizedquadrants of the original image, downloading each of the four resizedquadrants to the attachment viewer and assembling the resized quadrantsto display the full image of the original image on the portableelectronic device.

In another aspect there is provided a portable electronic deviceincluding: an attachment viewer stored on the portable electronicdevice, the attachment viewer for dividing an original image attachmentinto four quadrants and requesting resized quadrants to assemble anddisplay on the portable electronic device and an attachment server incommunication with the attachment viewer, the attachment server forbuilding a graph structure representing a map of the original imageattachment and sending the resized quadrants to the attachment viewer,wherein the resized quadrants when assembled appear as a single seamlessimage on a display of the portable electronic device.

Referring now to FIG. 1, a communication system 10 for a portableelectronic device 12 is generally shown. The portable electronic device12 is operable to effect communications over a radio communicationschannel and communicates with a base station (not shown) while locatedwithin a coverage area that is defined by the base station. The basestation is part of a wireless network that is in communication with theInternet 14. Data is delivered to the portable electronic device 12 viawireless transmission from the base station. Similarly, data is sentfrom the portable electronic device 12 via wireless transmission to thebase station.

It will be appreciated that the portable electronic device 12 is movablewithin the coverage area and can be moved to coverage areas defined byother base stations. Further, as will be understood by one of ordinaryskill in the art, wireless networks include GSM/GPRS, CDPD, TDMA, iDENMobitex, DataTAC networks, EDGE or UMTS and broadband networks such asBluetooth and variants of 802.11.

A server 18 handles wireless client requests from the portableelectronic device 12. A firewall, or proxy server, 16, is providedbetween the server 18 and the Internet 14. The server 18 furtheroperates as an attachment server, which communicates with an emailclient and an attachment viewer of the portable electronic device 12 toallow a user to view attachments that are received in email messages.While only one server 18 is shown for illustration purposes, a personskilled in the art will understand that the attachment server mayalternatively be a separate server.

Referring now to FIG. 2, a block diagram of certain components withinthe portable electronic device 12 is shown. In the present embodiment,the portable electronic device 12 is based on the computing environmentand functionality of a wireless personal digital assistant (PDA). Itwill be understood, however, that the portable electronic device 12 isnot limited to wireless personal digital assistants. Other portableelectronic devices are possible, such as smart telephones, and laptopcomputers.

The portable electronic device 12 is based on a microcomputer includinga processor 20 connected to a read-only-memory (ROM) 22 that contains aplurality of applications executable by the processor 20 that enableseach portable electronic device 12 to perform certain functionsincluding, for example, PIN message functions, SMS message functions andcellular telephone functions. The processor 20 is also connected to arandom access memory unit (RAM) 24 and a persistent storage device 26which are responsible for various non-volatile storage functions of theportable electronic device 12. The processor 20 receives input fromvarious input devices including a keypad 28. The processor 20 outputs tovarious output devices including an LCD display 30. A microphone 32 andphone speaker 34 are connected to the processor 20 for cellulartelephone functions. The processor 20 is also connected to a modem andradio device 36. The modem and radio device 36 is used to connect towireless networks and transmit and receive voice and data communicationsthrough an antenna 38. A content store 40, which is generally a filestorage system for the portable electronic device 12, is also provided.

Referring to FIG. 3, a method for viewing an attachment on portableelectronic device 12 according to an embodiment is generally shown.First (step 42), the attachment viewer displays a resized image of theattachment on the display 30 of portable electronic device 12. Anexample of a resized image 60 as it appears on the display 30 is shownin FIG. 4. The resized image 60 corresponds to a first page of a scanneddocument that the portable electronic device 12 received in an emailmessage in TIFF format. As shown, the text of the resized image 60 isdifficult to read due to the small size thereof.

Prior to step 42, the attachment server performs the resize operation onthe original image and downloads the resized image to the device 12. Inorder to perform the resize operation, the attachment server firstbuilds a Document Object Model (DOM) by parsing the attachment document.In this manner, a graph structure is built within the serverrepresenting a map of the original image. The original image is thenresized based on the requesting device image size limit, or devicescreen size width and height (in pixels). DOM structure is disclosed inUnited States Patent Application Nos. 2006/0055693 and 2002/0161796,which are herein incorporated by reference.

At step 44, the attachment viewer determines if the original image widthis greater than the device display width and/or if the original imageheight is greater than the device display height. If one or both of the,original image width and original image height are greater than thedisplay width and display height, respectively, then the attachmentviewer moves to step 46 and displays a “full image” option in anattachment viewer menu 62, which is shown in FIG. 4. If both the widthand height of the original image are less than those of the devicedisplay, the attachment viewer returns to step 42, the execution logicof FIG. 3 is terminated and the “full image” option does not appear inthe attachment viewer menu 62.

As indicated at step 50, if the user invokes the “full image” optionfrom the menu 62 (step 48), the attachment viewer first determines if afull image for the particular attachment was previously requested fromthe attachment server. Previously requested full images are storedlocally on the portable electronic device 12 in the content store 40.Therefore, the attachment viewer searches the content store 40 and ifcorresponding full image data is available for the requested image, theattachment viewer displays the full image, as indicated at step 56.

If no corresponding full image data is found, the attachment viewerrequests the “full image” from the attachment server. The original imageis divided into four quadrants, or tiles, by the attachment viewer witheach quadrant representing approximately ¼ of the original image. Therequest that is sent to the attachment server includes the device screensize as well as calculated width and height parameters of each of fourindividual quadrants of the original image, as indicated at step 52. Ifthe user does not invoke the “full image” option, the attachment viewerreturns to step 42 and simply continues to display the resized image.

If the attachment server successfully resizes each of the four quadrantsof the attachment image, the quadrants are returned to the attachmentviewer, assembled together and displayed on display 30 of the portableelectronic device 12 to provide the appearance of a single image, asindicated at step 56. At step 57, the quadrant data is saved in thecontent store 40 and then the execution logic of FIG. 3 is complete. Ifthe attachment server is unable to execute the “full image” option, anerror code is displayed instead, as indicated at step 58.

Successful execution of the “full image” option results in an imagebeing displayed in the attachment viewer that is larger and has a higherresolution than the image that is displayed at step 42. An example of alarger, higher resolution full image 64 is shown in FIG. 5. Once thefull image has been displayed, image manipulation operations areavailable to the user. The image appears seamless on the display 30 andmovement between quadrants is undetectable by the user. Imagemanipulation operations include zoom, pan, rotate, enhance and save, forexample. These operations are performed in a manner that is known in theart and generally include using a track wheel and arrow key, forexample, or another selection device of the portable electronic device12.

Referring now to FIG. 6, server side operation for the method forviewing an attachment on portable electronic device 12 of FIG. 3 isgenerally shown. At step 66, the attachment server receives the “FullImage” request from the attachment viewer of the portable electronicdevice 12. At step 68, the attachment server extracts the device screensize (DSS) and calculated width and height parameters of each quadrantfrom the attachment viewer request. The attachment server then retrievesthe DOM structure for the attachment from in-memory document DOM cache,as indicated at step 70. Once the DOM has been retrieved, the attachmentserver traverses through the DOM structure to determine thecorresponding image component in the DOM for that image, as indicated atstep 72. At step 74, the attachment server iterates through theattributes of the corresponding image component to determine if aseparate image component specifically for that screen size has alreadybeen constructed. The separate image component contains the same graphicinformation as the original image but is usually of a smaller size thanthe original image (generally less than or equal to the device screensize) and thus consumes much less memory (usually in ratio of 1:15 to1:20).

If the separate image component does not exist, the attachment servercollects the image binary data from the original image component inmemory, as indicated at step 76, and constructs a new image componentfrom the binary data, as indicated at step 78. At step 80, theattachment server caches the DSS image component as an attribute of theoriginal image component in the DOM structure. The DSS binary image datais collected at step 82 and then tailored to create new binary imagedata based on the calculated width and height parameters of each of thefour quadrants and the new binary image data is returned to theattachment viewer of the portable electronic device 12, as indicated atstep 84.

In operation, the user of the portable electronic device 12 receives anemail message having an attachment and opens the attachment in theattachment viewer. The attachment appears as shown in FIG. 4 withdetails that are difficult for the user to distinguish. If the originalattachment image is larger than the display 30 of the portableelectronic device 12, the “full image” option is available in theattachment viewer menu 62. The user then selects the “full image” optionand if the “full image” option has not been previously requested forthis particular image, the attachment viewer sends a request to theattachment server to provide a full image. The image is divided intofour quadrants and the attachment viewer request includes width andheight parameters for each of the four quadrants that are calculated bythe attachment viewer. The attachment server, upon receiving theattachment viewer request, constructs four new enlarged quadrants of theoriginal image and returns the four quadrants to the attachment viewer.The quadrants are then assembled together and the full image ispresented on display 30 and saved in the content store 40 of theportable electronic device 12.

A specific embodiment of the present embodiment has been shown anddescribed herein. However, modifications and variations may occur tothose skilled in the art. For example, although the portable electronicdevice 12 has been described having cellular telephone capabilities, thedescribed embodiment is not limited to portable electronic deviceshaving both cellular telephone and email capabilities. A portableelectronic device having email capabilities but not cellular telephonecapabilities may also be used. All such modifications and variations arebelieved to be within the sphere and scope of the present embodiment.

1. A method for viewing a full image of a server stored originalattachment on a portable electronic device comprising: building a graphstructure within said server representing a map of said original image;downloading and displaying a re-sized image from said server to saidportable electronic device, said original image exceeding an image sizelimit of said portable electronic device indicative of screen size ofsaid portable electronic device; dividing said original image into fourindividual quadrants and calculating width and height parameters of saidfour individual quadrants; sending a request from the portableelectronic device to the server to enlarge the image displayed on saiddevice based on said width and height parameters, said request includingsaid image size limit; retrieving and traversing said graph structurewithin the server to locate any corresponding image component for saidoriginal image and, upon locating said corresponding image componentiterating through attributes of the image component to determine if aseparate image component has been constructed for said image size limit;in the event that said separate image component has not been constructedthen collecting image binary data from said original image within saidserver, constructing a new image component from said binary data andcaching said new image component as an attribute of the original imagein said graph structure; collecting further image binary data from oneof either said new image component or said separate image component andmodifying said further binary data based on said width and heightparameters to provide four resized quadrants of said original image;downloading each of said four resized quadrants to said attachmentviewer; and assembling said resized quadrants to display said full imageof said original image on said portable electronic device.
 2. A methodas claimed in claim 1, wherein an original image width exceeds a widthof said image size limit and an original image height exceeds a heightof said image size limit.
 3. A method as claimed in claim 2, furthercomprising generating a menu selection at said portable electronicdevice for user enabling of said full image, said menu selection beinggenerated if said original image exceeds said image size limit.
 4. Amethod as claimed in claim 1, wherein said graph structure is a DocumentObject Model (DOM).
 5. A method as claimed in claim 1, furthercomprising storing said full image in memory of said portable electronicdevice.
 6. A method for viewing a full image of a server stored originalattachment on a portable electronic device comprising: building a graphstructure within said server representing a map of said original image;downloading and displaying a re-sized image from said server to saidportable electronic device, said original image exceeding an image sizelimit of said portable electronic device indicative of screen size ofsaid portable electronic device; dividing said original image into fourindividual quadrants and calculating width and height parameters of saidfour individual quadrants; sending a request from the portableelectronic device to the server to enlarge the image displayed on saiddevice based on said width and height parameters, said request includingsaid image size limit; retrieving and traversing said graph structurewithin the server to locate a separate image component constructed forsaid image size limit; collecting image binary data from said separateimage component and modifying said image binary data based on said widthand height parameters to create four resized quadrants of said originalimage; downloading each of said four resized quadrants to saidattachment viewer; and assembling said resized quadrants to display saidfull image of said original image on said portable electronic device. 7.A method as claimed in claim 6, wherein an original image width exceedsa width of said image size limit and an original image height exceeds aheight of said image size limit.
 8. A method as claimed in claim 6,further comprising storing said full image in memory of said portableelectronic device.
 9. A method as claimed in claim 6, wherein said graphstructure is a Document Object Model (DOM).
 10. A method as claimed inclaim 6, further comprising generating a menu selection at said portableelectronic device for user enabling of said full image, said menuselection being generated if said original image exceeds said image sizelimit.
 11. A portable electronic device comprising: an attachment viewerstored on said portable electronic device, said attachment viewer fordividing an original image attachment into four quadrants and requestingresized quadrants to assemble and display on said portable electronicdevice; and an attachment server in communication with said attachmentviewer, said attachment server for building a graph structurerepresenting a map of said original image attachment and sending saidresized quadrants to said attachment viewer; and wherein said resizedquadrants when assembled appear as a single seamless image on a displayof said portable electronic device.
 12. A portable electronic device asclaimed in claim 11, wherein said graph structure is a Document ObjectModel (DOM).